Skip to content

Touch over the USB link, touch as Dark Forces' mouse; the link from the shared emulink submodule - #24

Open
finger563 wants to merge 22 commits into
mainfrom
feat/remote-touch
Open

finger563 wants to merge 22 commits into
mainfrom
feat/remote-touch

Conversation

@finger563

@finger563 finger563 commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Also includes #25, merged into this branch: the USB link (devlink, the test control, emulink, the web console) moves to the shared esp-cpp/emulink submodule at components/emulink. It also includes a cppcheck fix: the cart pointer in TOUCH's frame mapping is now pointer-to-const.

Stacked on #23, which is merged; the PR now targets main. The same change for esp-box-emu is in its own PR.

Touch over the USB link

  • Protocol 5: TOUCH (0x0F): u8 fingers, u16 x, u16 y, u16 hold_ms. A finger is held for hold_ms and then lifted, or with 0 it stays down until the next TOUCH, so a drag is a run of them. fingers = 0 lifts it. The coordinates are screen pixels (1280×720). With flag 0x80 they are screenshot pixels instead; the firmware maps them through the frame's place on the screen.
  • Same as a real finger: the remote finger is treated like one everywhere. That covers touchpad_data() (ScummVM), the touch virtual gamepad and its menu hot corner, and the launcher's LVGL menus, which get a pointer input device of their own.
  • emulink: emulink tap X Y, emulink touch X Y / emulink touch (hold / lift), emulink drag X0 Y0 X1 Y1 --ms 300; --frame takes screenshot pixels, and --fingers 2 is ScummVM's right click.
  • Web console: the Screen panel's capture is a touch screen. Click or drag on it.

Touch as the mouse in Dark Forces

  • New API: Tab5Emu::touch_pointer(skip_gamepad) and screen_to_frame() give a game a pointer that can skip the touch gamepad's zones. Tab5Emu::frame_to_screen() is the inverse mapping.
  • Dark Forces: TheForceEngine feat/touch-mouse (6ef6cb8) uses them.
    • In the menus the finger is the cursor and a press is a left click. The touch gamepad's buttons don't count there.
    • In a mission, a drag off the touch gamepad is mouse look (yaw and pitch), and a quick tap fires.

Testing

Tried on the Tab5 with emulink:

  • Launcher: emulink tap on a game in the list selects it, and a tap on Play launches it.
    • Found here: LVGL rotates pointer input, so the remote finger is handed to LVGL in the panel's portrait coordinates. Fixed in c4ff9d5.
  • Dark Forces menus: emulink touch --frame 270 176 moves the cursor onto Begin Mission and clicks it. The mission briefing's OK works the same way.
  • Dark Forces mission: emulink drag in the middle of the screen turns (horizontal drag) and pitches (vertical drag); a quick emulink tap fires.
  • Not tried yet:
    • the web console's screen touch;
    • ScummVM with a remote touch, which reads the same touchpad_data().

🤖 Generated with Claude Code

https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY

finger563 and others added 18 commits November 14, 2026 10:27
…nd Dark Forces cores

The cores esp-box-emu moved to submodules (each repo holds the core plus
its port/ glue, written against BoxEmu, which the Tab5 aliases to
Tab5Emu), added the same way as the Genesis and PICO-8 ones:

- components/{nes,gbc,sms,msx,doom,darkforces}: the submodules plus their
  component CMake from the box, with the static state (.bss and .data)
  moved to PSRAM by tools/bss_to_psram.py (the P4 has no extern_ram
  placeholder for ldgen). fMSX and gnuboy define globals that snes9x, MGS,
  the Genesis core and the TPM runtime also define; those are renamed in
  their libraries with objcopy --redefine-syms.
- main/: the carts, ported from the box, with the Tab5's video settings
  (original = integer scale, fit, fill).
- rom_info: the new platforms (.nes, .gb, .gbc, .sms, .gg, .rom, .wad,
  DARK.GOB) and consoles (NES, Game Boy, Game Boy Color, Master System,
  Game Gear, MSX), in the box's order and colors.
- Tab5Emu: make_color(), osd_getromdata(), no-op haptics, and
  reserve_romdata(): MSX, Doom and Dark Forces run their allocator in the
  box's fixed 4 MB ROM block, which the Tab5 allocates per cart.
- pool_allocator from the box (JK's private pool functions are renamed
  jk_pool_* so they don't collide with it).

The core repos needed three RISC-V build fixes (fix/riscv-build in
esp-cpp/gnuboy, nofrendo and TheForceEngine): Xtensa-only inline asm and
an Xtensa-only ESP-IDF backtrace call.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
The NES ran at ~30 fps: the glue made 1600 audio samples a frame instead
of 367, and the Tab5 paces to the DAC. Measured after the fix: ~3 ms of
emulation per frame, 60 emulated / ~30 presented fps (nofrendo's frame
skip).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
- components/scummvm: esp-cpp/scummvm (branch esp32p4) as a submodule, with
  the ESP32 backend in backends/platform/esp32; built with the startrek
  engine only, no RTTI and no ScummVM GUI (~1.1 MB of flash).
- A game is a marker file *.scummvm in the game's folder (Emulator::SCUMMVM,
  shown under PC); ScummVM detects which game the folder holds.
- ScummvmCart: the engine runs in its own task; the pause menu parks it and
  saves / loads through the engine's save states into the slot files; the
  4:3 games' 320x200 is shown at 4:3. Touch is the mouse (two fingers = right
  click), as are a USB mouse and the gamepad (A / B = left / right click).
- Tab5Emu::display_size() getter (touch -> game coordinates).
- CI: checks out the new core submodules (ScummVM shallow).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
- esp-cpp/scummvm esp32p4 66a0f24: the backend's quit / load / task fixes
  and the startrek engine's save states and teardown fixes.
- The component renames ScummVM's allocation functions to the backend's
  PSRAM ones (backends/platform/esp32/psram_alloc.syms).

Tested on the Tab5 with the 25th Anniversary Enhanced CD (English + speech):
intro, bridge and briefings; pause-menu save (while the game waits for
input) and load (restarts the game with the state, from anywhere);
repeated launches and quits, including during the intro. Known: each engine
instance leaves ~90 KB of PSRAM behind (the engine's own teardown), until
the next boot.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
…tion

- esp-cpp/scummvm esp32p4 7c81792: three startrek engine leaks fixed (LZSS
  input streams, the bridge's ever-growing action queue, the save
  thumbnail) and the backend's detection data left on a jumped-over stack.
- idf.py -DSCUMMVM_TRACK_ALLOCS=1 build: tracks ScummVM's allocations and
  writes what each run leaves to /sdcard/saves/scummvm_leaks.txt.

Measured on the Tab5: free PSRAM flat during bridge play (was -600 B/s), and
a few hundred bytes a game run (was 50-130 KB).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
…oom's crash

- Dark Forces: its free() wrapper kept claiming its ended session's 4 MB
  block, which the Tab5 gives back to the heap with the cart: every free()
  the other games later made into that range was swallowed, and Jedi Knight
  hung for want of memory. The cart now calls darkforces_forget_block()
  (TheForceEngine 03417ff) before the block goes.
- ScummVM (esp-cpp/scummvm a0a35c3): a game runs in its own PSRAM arena,
  given back whole at its end, and the long-lived set-up is done at boot
  (scummvm_prepare() in main): Star Trek no longer leaves blocks mid-heap
  that split the big ones Jedi Knight wants.
- Tab5Emu::play_audio()'s 30 s report is debug-level: it is formatted on the
  game's audio task, and Doom's 2 KB one overflowed (a stack protection
  fault after 30 s of play).

Checked on the Tab5: Doom (60+ s), Dark Forces, Star Trek, then Jedi Knight
(20 fps), MGS, TPM, GBA, Genesis, PICO-8, NES, GB/GBC, SMS/GG, MSX.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
…d fails late)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
…-op here)

Built without exceptions (ESP-IDF), libstdc++'s vector::shrink_to_fit()
does nothing: Jedi Knight's presentation buffers (~1.8 MB: the RGB565 frame,
two 8-bit frames, two overlays) and TPM's copy of the game executable stayed
allocated after the game. With ~1.35 MB less PSRAM since the new cores and
ScummVM, the SNES core then had too little left after Jedi Knight. They are
swapped with empty vectors now.

Checked on the Tab5: free PSRAM after quitting Jedi Knight is back to what
it was before (7.28 MB, largest block 7.17 MB); JK -> SNES -> TPM -> SNES ->
JK all run.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
Bumps the esp-cpp/scummvm fork: config defaults registered (speech and
effects played at volume 0 once the engine synced its settings), repeated
log lines counted (the per-row drawR3Shape warning stalled the intro), close
shapes drawn and shapes behind the camera culled (the intro's planet).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
Bumps the esp-cpp/scummvm fork to the reverse-engineered ship rendering
(views picked by angle, the original's 2D transform, dithered close-ups),
the view check and draw order, and the intro's movement, credits and
captions as the original plays them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
Bumps the esp-cpp/scummvm fork: the bridge's viewscreen shows space, the
star systems' planets (generated as the original does), entering and
leaving orbit and the mission start fly-bys as the original plays them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
…is known

The ScummVM cart applied fit/original before the engine had set up its
screen (0x0 then, ScummVM being prepared at boot), so a game started
filling the screen until the pause menu re-applied it. It now follows the
screen's size whenever that changes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
Bumps the fork: ships fly and fight as the original's, the captain flies
and fires from the viewscreen (tap: phasers, two fingers: torpedoes), and
the missions' bridge scripts with their battles.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
…orces' mouse

The link's test control gets TOUCH (protocol 5): a finger at (x, y), held
for a time or until the next TOUCH, in the screen's pixels or (flag 0x80)
the screenshot's, which the firmware maps through the frame's place on
the screen. The finger counts as a touch everywhere: touchpad_data()
(ScummVM), the touch virtual gamepad and its menu hot corner, and the
launcher's LVGL menus (a pointer input device of its own).

emulink: tap, touch (hold / lift) and drag, --frame for screenshot
pixels. The web console's screen is a touch screen: click or drag on it.

Tab5Emu::touch_pointer() / screen_to_frame() give games a pointer that
can skip the touch gamepad's zones; Dark Forces (TheForceEngine
feat/touch-mouse) uses them: in the menus the finger is the cursor, in a
mission a drag off the touch gamepad is mouse look and a quick tap fires.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
LVGL rotates what a pointer input device reads (the BSP's touch is the
panel's portrait coordinates), so the link's finger, in landscape, landed
elsewhere: rotate it back first.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
@finger563
finger563 marked this pull request as ready for review November 16, 2026 00:35
…ariablePointer)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY
Base automatically changed from feat/box-cores to main November 16, 2026 02:33
finger563 and others added 2 commits November 15, 2026 20:33
components/devlink, main/remote_control.hpp, tools/emulink and web/ were
the same in esp-box-emu and tab5-emu; they are esp-cpp/emulink now, a
submodule at components/emulink (the `emulink` component: DevLink and
the header-only RemoteControl). tools/emulink and web are links into it,
so `cd tools/emulink && python -m emulink` and the local web server work
as before; the pages workflow publishes components/emulink/web. The
board component comes from EMULINK_BOARD_COMPONENT (box-emu by default).


Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
# Conflicts:
#	.gitmodules
#	CMakeLists.txt
#	components/darkforces/TheForceEngine
@finger563 finger563 changed the title Touch over the USB link (emulink tap / touch / drag); touch as Dark Forces' mouse Touch over the USB link, touch as Dark Forces' mouse; the link from the shared emulink submodule Nov 16, 2026
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SFCMzomCZ5wRHJN4sxwkvY

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant